1. Field of the Invention
This invention relates to the arts of imaging and image processing, including sensor design and moiré reduction technologies.
2. Background of the Invention
Well-known imaging technologies include film (e.g. analog), digital, and analog-digital hybrid approaches. Film imaging processes use a set of lenses to focus an image onto a film sheet which is impregnated with grains of material reactive to the spectrum to be recorded, such as visible light, infrared (“IR”), or X-ray. The grains are randomly arranged in each sheet of film, and thus reproduction of the image on the developed film has a certain resolution based on the size and density of these grains.
In digital imaging, a sensor of uniformly-arranged sensing elements is used to capture “bits” or pixels of the image. Turning to FIG. 1, the system components (10) of a typical digital camera are shown. In this example, a scene or original item (1) is digitally imaged using a two-dimensional array (7) of sensors such as an array of charge-coupled devices (“CCD”). The image of the scene is focused onto the array (7) by a lens (6), and a shutter (not shown) may be used to provided a specific duration of exposure. The sensor elements are arranged with uniform spacing into rows and columns.
Turning to FIG. 2, more details of a typical two-dimensional sensor array (7) are shown. The sensor element columns are uniformly spaced at distance d1 from each other according to a linear function such as:x-axis position of sensor in column n=Pn=(n−1)·d1 where the array is comprised of N sensor columns, n is the column number ranging from 1 through N, and d1 is the uniform distance between the sensor columns.
Likewise, the sensor element rows are uniformly spaced at distance d2 from each other according to a linear function such as:y-axis position of sensor in column m=Pm=(m−1)·d2 where the array is comprised of M sensor rows, m is the row number ranging from 1 through M, and d2 is the uniform distance between the sensor rows.
In many two-dimensional sensor arrays, the row-to-row spacing d1 may be equal to the column-to-column spacing d2. The number of columns N may be equal to the number of columns M, as well.
The sensors may in practice be reactive to any range of electromagnetic (“EM”) spectrum according to the desired application, such as charge-coupled devices (“CCD”) for visible or IR imaging.
Typically, the voltage level on each sensor element is measured and converted (e.g. sampled) to a digital value using an analog-to-digital converter. The sample value is relative to the amount of electromagnetic energy incident on the sensor element. Conversion to digital values are typically performed using an analog-to-digital converter having sufficient resolution (e.g. data width) for the intended application. The digital data set (66) of samples represents a digitized or pixelated copy of the image.
Additionally, mechanical and/or chemical filtering and band separation of the EM spectrum may be performed to produce “separated” data sets, such as use of a color wheel in front of the sensor array, or placement of color filters over the sensor elements themselves.
Moiré patterns are artifacts of certain imaging processes which are perceptible to the human eye, but do not represent actual features or details in the original item imaged. They often resemble crosshatch halftones across all or a portion of a digital image.
For imaging processes in which the original is an analog image, for example a photographic subject, moiré patterns may appear when certain features align with the sensors in the sensor array. For example, a digital photograph of a bug screen on a window often produces noticeable moiré patterns due to the bug screen's uniform grid-like features. The resulting apparent pattern is actually an interference pattern between the physical spacing of features of the original image and the spacing of the sensor array.
Just as in the phenomena of interference patterns between other types of signals, visual moiré patterns may become apparent at “harmonics” or integral multiples of spacing distances of the original image features and the spacing distances of the sensor array. For example, if the repeating features of a photographic subject are focussed onto two-dimensional sensor array having a sensor spacing of 600 dots per inch (“DPI”) and a moiré pattern forms, then the same image focused at the same distance on a sensor array having a sensor spacing of 1200 DPI will likely result in the appearance of moiré patterns. Undersampling the image at 300 DPI would also likely result in the appearance of moiré patterns.
Many techniques have been developed to try to reduce moiré patterns which appear in existing digital images, such as application of image processing techniques including Gaussian blurring, “descreening” algorithms, and “de-speckle” processes. Most of these have a result of reducing the sharpness of the overall image because they reduce the moiré pattern by spreading energy or brightness from a given pixel to adjacent pixels.
For example, turning to FIG. 3a, pixel N represents a pixel of a moiré pattern in a single row or column, and in this case, a pattern which is darker than the surrounding pixels, N−1 and N+1. The energy E2 of pixel N is lower than the energy E3 of the adjacent pixels N+1 and N−1. A blurring process applies a partial or weighted averaging among regional or adjacent pixels, such as shown in FIG. 3b, wherein the energy of the pixel in the moiré pattern is slightly increased to E2′, and the energy of the adjacent pixels are slightly decreased to E3′.
While this oftentimes decreases the obviousness or appearance of the moiré pattern to the human observer, it also reduces the “sharpness” or level of apparent detail of the entire image. If the blurring process is applied manually on a regional basis, the degradation to the entire image may be avoided, but the local areas are still degraded and substantial human intervention may be required to do so. Additionally, “edge effects” may become perceptible where the region of processing meets a region of unprocessed image.
So, to date, most digital image post-processing attempts to reduce moiré patterns either result in image degradation, require substantial human operator effort, or both to some degree.
A common technique employed to avoid the generation of moiré patterns in the imaging process is to dither the sensor array such that the array is moved in physical position with respect to the original subject being imaged. In FIG. 4, such a dithering imaging system with a two-dimensional sensor array (7) is shown. An x-axis mechanical jitter drive (40) is coupled to the array (7) such that it's x-axis position is varied slightly over time, usually in a sinusoidal or triangular pattern (41). Likewise, a y-axis jitter drive (42) may jitter the array in an orthogonal direction, also typically in a sinusoidal or triangular patter (43).
This jittering action allows the array (7) to scan a pattern of points which are not simply an array of uniformly spaced rows and columns, but which represent positions relative to the dithering functions Px′(t) and Py′(t). As such, fewer original image sources will have an interference pattern with the dithered sensor pattern, but it is still possible that portions of the original image source may interfere with the dithered sensor pattern to cause localized moiré patterns. Additionally, such dithering mechanisms tend to add expense and failure rate to an assembly such as a digital camera.
Therefore, there is a need in the art for a system and method which avoids generation of moiré patterns in digital images created with a two-dimensional sensor array, without the use of mechanical dithering mechanisms, intensive image post-processing technologies, or a high degree of human operator manipulation and editing. Further, there is a need in the art for this new system and method to maintain image quality, while being readily realizable using current sensor technology, and to preferrably be compatible with widely-used image compression and decompression technologies such as bitmap, JPEG (joint photographic experts) and MPEG image products.